Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add email domain filter #98

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jdmulloy
Copy link

@jdmulloy jdmulloy commented Feb 1, 2018

This is useful for operators who have users whose Github accounts have multiple email addresses and their default address is a personal address. The current implementation leads to Jenkins storing the default github address, which in some cases is a personal account. This change allows an operator to specify the domain of their org, and the plugin will look at all the emails for a user and if it finds one that matches the specified domain it will prefer that email over the primary. If the setting is empty it will behave as before.

I also added a checkbox to always override the stored value with what is found in Github.

This is very much a work in progress. I'm not an experienced Java developer and I did this in one working day. It probably also needs some tests for the new functionality. I'd appreciate feedback.

@samrocketman
Copy link
Member

This makes sense. I'll take a look.

@samrocketman
Copy link
Member

@jenkinsci/code-reviewers Please check out this change.

@samrocketman
Copy link
Member

@jdmulloy mind if I make a feature request? A CSV list of domains in order of preference. Some companies have contractors who would be using contractor emails. If not, that's fine, I could try adding it later.

@jdmulloy
Copy link
Author

jdmulloy commented Feb 7, 2018

@samrocketman I've added in some logging and added the csv feature.

@samrocketman
Copy link
Member

Thanks for humoring my request @jdmulloy. I'll test running and upgrading to your version to ensure stability. If all goes well I'll merge it. I think this is a pretty cool feature.

}

writer.startNode("forceGithubEmail");
//TODO: Is there a better way to do this?
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a better/preferred way to store a boolean value in the config file? If not and what I have is ok I'll remove the TODO comment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check how other plugins save and load booleans. I'll check as well.

@jdmulloy
Copy link
Author

jdmulloy commented Feb 8, 2018

Once it's approved I can squash the commits to make the history cleaner.

@samrocketman
Copy link
Member

I have not forgotten about this change. I was going to look at it over the weekend and did not get a chance. I'll see about looking at it during the evenings this week.

String oauthScopes) {
String oauthScopes,
String emailDomains,
Boolean forceGithubEmail) {
Copy link
Member

@samrocketman samrocketman Feb 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not cleanly migrate configuration. Please overload this method with another deprecated method so that configurations cleanly migrate.

Copy link
Member

@samrocketman samrocketman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert the old failed test

I accidentally advised you to update a test rather than address its failure. A mistake on my part.

  • Please revert the change so the tests fail again.
  • Update your PR so the tests pass.
  • Add a few additional tests to the existing tests so that your change is properly tested.

I can help you with this if you need it.

Fix configuration migration

This will not cleanly migrate configuration. Please overload GithubSecurityRealm() constructor with another deprecated constructor so that configurations cleanly migrate.

Note: this should fix existing failed tests.

A simple test:

  1. Build this plugin.
  2. Start Jenkins with the stable GitHub OAuth plugin configured.
  3. Upgrade to this built plugin and see how the configuration migration behaves.

I'll take a closer look at this during the week. There are other plugins which do as I describe. Try to find examples there. Otherwise, I'll try to find an example.

Example:

/*
   This method is deprecated.
   @deprecated use GithubSecurityRealm(githubWebUri, githubApiUri, clientID, clientSecret, oauthScopes, emailDomains, forceGithubEmail)
 */
@Deprecated
public GithubSecurityRealm(String githubWebUri,
             String githubApiUri,
             String clientID,
             String clientSecret,
             String oauthScopes) {
    GithubSecurityRealm(githubWebUri, githubApiUri, clientID, clientSecret, oauthScopes, "", false);
}

Fix object equality

/**
* Compare an object against this instance for equivalence.
* @param object An object to campare this instance to.
* @return true if the objects are the same instance and configuration.
*/
@Override
public boolean equals(Object object){
if(object instanceof GithubSecurityRealm) {
GithubSecurityRealm obj = (GithubSecurityRealm) object;
return this.getGithubWebUri().equals(obj.getGithubWebUri()) &&
this.getGithubApiUri().equals(obj.getGithubApiUri()) &&
this.getClientID().equals(obj.getClientID()) &&
this.getClientSecret().equals(obj.getClientSecret()) &&
this.getOauthScopes().equals(obj.getOauthScopes());
} else {
return false;
}
}

^^ the above link uses an "equals()" method. You're adding new properties to the object without testing for equality. That means if there's differences in your settings, the equals() method will misreport that the two objects are equal in terms of property values.

@jdmulloy
Copy link
Author

@samrocketman I'll work on overloading the method and fixing the tests. How do I run the tests that are failing? When I build the plugin I don't get any test failures, so there is probably something I'm missing. I can't find any failing tests via this PR, but I may not know where to look.

Thanks

@jdmulloy
Copy link
Author

Actually I think I figured out what you were saying. You were talking about the tests that were failing until I changed them. Correct?

@jdmulloy
Copy link
Author

@samrocketman I added the deprecated method, but it fails to compile with the following error. I can't figure out why it can't find the method. I've tried googling.

[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ github-oauth ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.3:compile (default-compile) @ github-oauth ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 11 source files to /home/jdmulloy/git/github-oauth-plugin/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /home/jdmulloy/git/github-oauth-plugin/src/main/java/org/jenkinsci/plugins/GithubSecurityRealm.java:[164,9] cannot find symbol
  symbol:   method GithubSecurityRealm(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.Boolean)
  location: class org.jenkinsci.plugins.GithubSecurityRealm
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.973 s
[INFO] Finished at: 2018-02-14T12:36:43-05:00
[INFO] Final Memory: 41M/155M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.3:compile (default-compile) on project github-oauth: Compilation failure
[ERROR] /home/jdmulloy/git/github-oauth-plugin/src/main/java/org/jenkinsci/plugins/GithubSecurityRealm.java:[164,9] cannot find symbol
[ERROR]   symbol:   method GithubSecurityRealm(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.Boolean)
[ERROR]   location: class org.jenkinsci.plugins.GithubSecurityRealm
[ERROR] 
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

}

@Test
public void testHasScope_true() {
GithubSecurityRealm a = new GithubSecurityRealm("http://jenkins.acme.com", "http://jenkins.acme.com/api/v3", "someid", "somesecret", "read:org,user,user:email", "", false);
GithubSecurityRealm b = new GithubSecurityRealm("http://jenkins.acme.com", "http://jenkins.acme.com/api/v3", "someid", "somesecret", "read:org,user,user:email");
GithubSecurityRealm a = new GithubSecurityRealm("http:jenkins.acme.com", "http://jenkins.acme.com/api/v3", "someid", "somesecret", "read:org,user,user:email");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to remove // from http://? This comment applies to other parts as well.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching that. It was an accident. I had commented out a bunch of lines while debugging and used a regex to remove the comments.

@samrocketman
Copy link
Member

Let me know when your'e ready for me to review/test this change for merging.

@samrocketman
Copy link
Member

samrocketman commented Feb 18, 2018

Please squash and rebase on master.

@jdmulloy jdmulloy force-pushed the jdmulloy/add_email_domain_filter branch from c23a1da to 793d598 Compare February 23, 2018 19:43
@jdmulloy
Copy link
Author

@samrocketman I've squashed and rebased on master. I tested upgrading the plugin and it works now. I had to fix handling of the forceGithubEmail setting being missing in config.xml. The unmarshaling code was setting it to Null instead of False.

The todo comment for handling the boolean forceGithubEmail is still there. The useSecurity setting in core uses "true" in config.xml to store true, but I couldn't find the relevant marshaling and unmarshaling code for that. So I couldn't figure out how that's done in core.

@samrocketman
Copy link
Member

Thanks I'll look it over and test it.

@samrocketman
Copy link
Member

Upgraded by uploading via the Advanced and checking the restart Jenkins checkbox. Jenkins threw a stacktrace upon upgrade.

Click here for stack trace
java.lang.NullPointerException
	at com.thoughtworks.xstream.io.xml.PrettyPrintWriter.writeText(PrettyPrintWriter.java:235)
	at com.thoughtworks.xstream.io.xml.PrettyPrintWriter.writeText(PrettyPrintWriter.java:231)
	at com.thoughtworks.xstream.io.xml.PrettyPrintWriter.setValue(PrettyPrintWriter.java:214)
	at com.thoughtworks.xstream.io.WriterWrapper.setValue(WriterWrapper.java:45)
	at org.jenkinsci.plugins.GithubSecurityRealm$ConverterImpl.marshal(GithubSecurityRealm.java:285)
	at hudson.util.XStream2$AssociatedConverterImpl.marshal(XStream2.java:370)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
	at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265)
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252)
Caused: java.lang.RuntimeException: Failed to serialize jenkins.model.Jenkins#securityRealm for class hudson.model.Hudson
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:256)
	at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:224)
	at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
	at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)
	at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
	at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37)
	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026)
	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015)
	at com.thoughtworks.xstream.XStream.toXML(XStream.java:988)
	at hudson.XmlFile.write(XmlFile.java:181)
	at jenkins.model.Jenkins.save(Jenkins.java:3198)
	at jenkins.model.Jenkins.saveQuietly(Jenkins.java:3204)
	at jenkins.model.Jenkins.setSecurityRealm(Jenkins.java:2576)
	at jenkins.model.Jenkins$17.run(Jenkins.java:3168)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1068)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: org.jvnet.hudson.reactor.ReactorException
	at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:269)
	at jenkins.InitReactorRunner.run(InitReactorRunner.java:47)
	at jenkins.model.Jenkins.executeReactor(Jenkins.java:1102)
	at jenkins.model.Jenkins.<init>(Jenkins.java:904)
	at hudson.model.Hudson.<init>(Hudson.java:86)
	at hudson.model.Hudson.<init>(Hudson.java:82)
	at hudson.WebAppMain$3.run(WebAppMain.java:233)
Caused: hudson.util.HudsonFailedToLoad
	at hudson.WebAppMain$3.run(WebAppMain.java:250)

To reproduce this issue I used the my jenkins-bootstrap-jervis project which you can use settings.groovy to configure the GitHub OAuth plugin. Uncomment the bottom of settings.groovy to configure github-oauth.

dependencies.gradle of jenkins-bootstrap-jervis will show you what versions of plugins I tested.

@samrocketman
Copy link
Member

However, when I restarted Jenkins the stacktrace went away.

@samrocketman
Copy link
Member

I'm going to investigate this more before merging.

@samrocketman
Copy link
Member

Also note, if you wanted to test this yourself without the complication of learning the scripts I publish packages of the version I tested. https://github.com/samrocketman/jenkins-bootstrap-jervis/releases/tag/jervis-bootstrap-2.89.4.3

@samrocketman
Copy link
Member

samrocketman commented Dec 7, 2018

@jdmulloy I just released a bunch of security fixes and a critical bug fix. Unfortunately, I didn't get a chance to test your change as part of the release. Would you mind merging in the latest master and resolving the merge conflicts?

If not, then I can take a look and update it after I move.

@scurvydoggo scurvydoggo requested a review from a team as a code owner July 31, 2023 04:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants